Bag of Surrogate Parts: one inherent feature of deep CNNs

In this paper, we first develop a new feature from the last pooling layer (i.e. pool5) of VGG, called Bag of Surrogate Parts (BoSP), and its spatial variant, Spatial BoSP (S-BoSP). Next, we propose a scale pooling scheme for better handling the objects that may appear in different shape, positions and scales. Finally, aiming that the traditional data augmentation focuses more on part the original image, we further raise a global constrained augmentation method to make a more comprehensive prediction. The details of our contributions are described below: Bag of Surrogate Parts (BoSP) We take the feature maps as surrogate parts and assume that the activation values represent the assignment strengths for these parts. Therefore, given the architecture, the number of the surrogate parts is inherently determined, as is the same with the number of feature maps. For each spatial unit, we calculate its assignment strengths for the surrogate parts by observing its activation values. The one-by-one processing of these spatial units can be viewed as densely sampling and assigning regions of the input image. Finally, we sum the assignment strengths for the surrogate parts and form a vector accordingly, i.e. BoSP, whose length is the same with the number of the feature maps. The framework of the proposed BoSP feature is shown in Figure 1. Specifically, the BoSP for this image can be represented as Eq.(1):